WebGPU: Add indirect dispatch for flash attention graph capture to support Gemma4 by feich-ms · Pull Request #29236 · microsoft/onnxruntime

feich-ms · 2026-06-23T10:28:38Z

Summary

This adds indirect dispatch support for WebGPU flash attention to enable graph capture for Gemma4. When graph capture is on, `total_seqlen` is GPU-resident and cannot be read on the CPU, so dispatch group sizes must be computed on GPU.

CopyKVCache normally prepares the indirect dispatch buffer as a side effect. For Gemma4 kv_empty layers (shared KV layers that skip CopyKVCache), a dedicated `PrepareIndirectDispatchProgram` fills the indirect buffer instead — avoiding `dispatch(0)` crashes.

Adds `PrepareIndirectDispatchProgram`: a single-thread GPU kernel that reads `total_sequence_length_input[0]` and writes 3 x uint32 dispatch dims to `indirect_buffer`, matching the logic in `CopyKVCacheProgram`
`use_indirect_dispatch` is gated on `context.IsGraphCaptureEnabled()` — indirect dispatch is only needed when `total_seqlen` is GPU-resident; when graph capture is off, CPU-side dispatch sizing works correctly even for kv_empty layers
`PrepareIndirectDispatchProgram` takes `total_sequence_length_input` (the global max, batch-safe) rather than `seqlen_k` (per-batch index 0, unsafe for batch > 1)

Changes

`flash_attention.cc`:
- New `PrepareIndirectDispatchProgram::GenerateShaderCode`: reads `total_sequence_length_input[0]`, computes tile count, calls `populate_indirect_dispatch_buffer` — identical logic to `CopyKVCacheProgram`'s indirect dispatch block
- `use_indirect_dispatch` gated on `seqlen_k != nullptr && total_seqlen != nullptr && context.IsGraphCaptureEnabled()`
- kv_empty path calls `PrepareIndirectDispatchProgram` when `use_indirect_dispatch` is true (i.e., under graph capture)
`flash_attention.h`: Added `PrepareIndirectDispatchProgram` class declaration

Test plan

Verified with Gemma4 INT4 model: ~95-105 tok/s with GC=ON vs ~75 tok/s with GC=OFF
Tested multiple prompts (short/long) with graph capture enabled — no crashes, correct output
Verified warmup (capture run) followed by replay produces consistent results
Unit test: `WebGPU_SharedKV_IndirectDispatchForGraphCapture` — exercises the full ORT graph capture simulation path end-to-end:
- Model built via the Graph API (opset 17) with all GQA inputs declared as proper graph inputs
- All inputs allocated as GPU-resident tensors via `InferenceSession::GetAllocator` + `IOBinding`; `total_seqlen` is a real `WGPUBuffer` so `PrepareIndirectDispatchProgram` reads it from GPU
- WebGPU EP registered with `kEnableGraphCapture=ON`; first `Run` captures, second `Run` replays
- Replay output copied GPU→CPU and compared against a CPU reference to verify correctness
- Covers the kv_empty branch specifically: `key`/`value` have `sequence_length=0`, triggering the `PrepareIndirectDispatchProgram` code path (CopyKVCache is skipped for kv_empty layers)

Copilot

Pull request overview

This PR improves the WebGPU FlashAttention decode path’s compatibility with graph capture and dynamic sequence lengths by enabling GPU-side computation of indirect dispatch group sizes from seqlen_k (including the kv_empty/shared-KV case that previously could produce dispatch(0)).

Changes:

Added a dedicated PrepareIndirectDispatchProgram shader to compute normalized indirect dispatch dimensions on GPU from seqlen_k.
Expanded use_seqlen_k / use_indirect_dispatch gating so indirect dispatch is also available when total_sequence_length_ is unavailable on CPU (including kv_empty layers).
Ensured the kv_empty path prepares the indirect dispatch buffer even when CopyKVCache is skipped.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
onnxruntime/contrib_ops/webgpu/bert/flash_attention.h	Declares `PrepareIndirectDispatchProgram` and its uniform interface.
onnxruntime/contrib_ops/webgpu/bert/flash_attention.cc	Implements the new program and wires indirect-dispatch preparation into the `kv_empty` path and broadened gating logic.

github-actions

You can commit the suggested changes from lintrunner.

- Extract AppendNormalizeDispatchShader() helper so CopyKVCacheProgram and PrepareIndirectDispatchProgram share one copy of the tile-count and normalize_dispatch_group_size call instead of duplicating it. - Express use_indirect_dispatch as use_seqlen_k && (share_buffer || kv_empty) to make the subset relationship explicit and eliminate the repeated condition. - Tighten use_seqlen_k / use_indirect_dispatch guards from == 0 to <= 0 to handle a negative total_sequence_length_ defensively. - Add comment on the WGSL template's normalize call pointing back to the C++ helper so the two stay in sync. Co-Authored-By: Claude <noreply@anthropic.com>

Add two WebGPU GQA tests that exercise PrepareIndirectDispatchProgram: - WebGPU_SharedKV_IndirectDispatch_Decode: kv_empty + total_sequence_length=0 (decode, past_seq=8), triggers use_seqlen_k=true and use_indirect_dispatch=true via the kv_empty path, cross-checked against CPU reference. - WebGPU_SharedKV_IndirectDispatch_LargerPast: same path with past_seq=32 to exercise num_total_seq_length_tile > 1 in the tile count arithmetic. Co-Authored-By: Claude <noreply@anthropic.com>

Co-Authored-By: Claude <noreply@anthropic.com>

Replace deprecated bool use_cuda/use_webgpu params with GqaTargetEp::kCpu. Co-Authored-By: Claude <noreply@anthropic.com>

OpTester cannot enable graph capture, so use_indirect_dispatch is never triggered. Rewrite the tests to exercise the kv_empty path directly with a real positive total_sequence_length instead of 0. Co-Authored-By: Claude <noreply@anthropic.com>

Gemma4 layers 15-34 are KV-sharing layers (kv_sequence_length==0): they reuse KV from a different layer's output, so past/present cannot share the same GPU buffer even with past_present_share_buffer=true in the config. The assertion added in 92b4c66 correctly enforces that graph capture requires static KV cache (shared buffers), but must exempt kv_empty layers since their cross-layer KV sharing is expected and valid. Move kv_empty definition before the ORT_ENFORCE so it can be used directly in the condition rather than inlining kv_sequence_length_ == 0. Co-Authored-By: Claude <noreply@anthropic.com>

AppendNormalizeDispatchShader referenced the old constant and WGSL function names from before main renamed them to kPopulateIndirectDispatchBufferFn / populate_indirect_dispatch_buffer. Co-Authored-By: Claude <noreply@anthropic.com>

…tDispatchShader Aligns the helper name with the underlying WGSL function and constant it emits (populate_indirect_dispatch_buffer / kPopulateIndirectDispatchBufferFn) after main renamed these from the old normalize_dispatch_group_size naming. Co-Authored-By: Claude <noreply@anthropic.com>

…l site Single-use static helpers add indirection without benefit. Co-Authored-By: Claude <noreply@anthropic.com>

Addresses reviewer feedback: use total_sequence_length_input (the global max, batch-safe) instead of seqlen_k[0] (per-batch index 0, unsafe for batch > 1). Aligns naming and logic with CopyKVCacheProgram which reads the same input the same way. Also clarifies comments to make explicit that indirect dispatch is only needed under graph capture, when total_seqlen is GPU-resident and CPU-side dispatch sizing is unavailable. Co-Authored-By: Claude <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

qjia7

Summary

This PR enables WebGPU graph capture for Gemma4 by adding a dedicated
PrepareIndirectDispatchProgram for kv_empty (shared-KV) layers, which skip
CopyKVCache and would otherwise leave the indirect dispatch buffer
uninitialized. The earlier review thread (qjia7) has been addressed: the new
shader reads total_sequence_length_input[0] (global max across the batch),
not seqlens_k[0], so it is safe for batched right-padded prefill.

Findings

Q1 — Test duplication of `RunGQASharedKV`

WebGPU_SharedKV_KvEmpty_Decode and WebGPU_SharedKV_KvEmpty_LargerPast
are each ~60 lines and inline the entire OpTester setup that
RunGQASharedKV (in the same file) already provides. The existing helper
takes a GqaTargetEp argument and is used elsewhere with kCpu /
kWebGpu / kCuda. The two new tests should reduce to:

auto webgpu_output = RunGQASharedKV(batch_size, q_seq_len, past_seq_len,
                                    query_data, past_key_data, past_value_data,
                                    num_heads, kv_num_heads, head_size,
                                    GqaTargetEp::kWebGpu);
auto cpu_output    = RunGQASharedKV(/* ...same args... */, GqaTargetEp::kCpu);
ExpectOutputsMatch(webgpu_output, cpu_output, 0.05f, "...");

The two new tests then differ only in past_seq_len (8 vs 32) and can be
parameterized.

Q2 — The new shader is not exercised by either test

use_indirect_dispatch requires context.IsGraphCaptureEnabled(), which
OpTester does not toggle. As written, both new tests run the
non-graph-capture path — they cover the kv_empty aliasing branch but
not PrepareIndirectDispatchProgram itself. The shader is currently
covered only by the end-to-end Gemma4 run mentioned in the PR
description, which is not reproducible from a unit test.

Please extend the WebGPU test harness so OpTester (or a sibling helper)
can opt into a graph-capture session option, then add a test that
actually drives PrepareIndirectDispatchProgram. Without this, any
future refactor of the new shader has no automated guard.

Verdict

Approve with changes. Pre-merge: Q1 (test refactor to use
RunGQASharedKV) and Q2 (wire graph capture into the WebGPU test
harness and add coverage for PrepareIndirectDispatchProgram) both need
attention before merge — Q2 is what gives the new shader real test
coverage.

The correctness story is sound: total_seqlen is the right input for
indirect-dispatch sizing, the kv_empty aliasing already existed and is
unchanged, and the ORT_ENFORCE relaxation is the smallest possible
condition under which Gemma4 layers can pass.

Adds WebGPU_SharedKVLayers_IndirectDispatchForGraphCapture to verify PrepareIndirectDispatchProgram correctly computes GPU-side dispatch sizes for kv_empty (shared-KV) layers when graph capture is enabled and total_seqlen is GPU-resident. Uses gpu_graph_id=-1 to keep IsGraphCaptureEnabled()=true without triggering actual Dawn capture/replay. Co-Authored-By: Claude <noreply@anthropic.com>

Build the model via Graph API (opset 17) instead of OpTester::BuildModel so graph inputs are properly declared after graph.Resolve(). Bind all inputs as GPU-resident OrtValues via IOBinding: PrepareIndirectDispatchProgram now reads total_seqlen from a real WGPUBuffer instead of a CPU pointer cast, exercising the full capture-then-replay path as the reviewer requested. Co-Authored-By: Claude <noreply@anthropic.com>

…rectness Add a second input set (set B) with distinct values. For the replay run, copy set B into the same GPU buffers via CopyTensor — rebinding is not allowed since WGPUBuffer pointers are baked in at capture time. Verify the replay output matches the CPU reference for set B, not set A, proving the captured graph actually executes with the updated data. Co-Authored-By: Claude <noreply@anthropic.com>

Co-Authored-By: Claude <noreply@anthropic.com>

hariharans29 · 2026-07-02T21:37:23Z

Review: PR #29236 — WebGPU: Add indirect dispatch for flash attention graph capture to support Gemma4

LGTM. The scope is right, the fix is minimal, and the rewritten test now genuinely exercises the graph-capture path — which was the outstanding review ask before this round. A few drive-by nits below, all non-blocking.

What's good

The math in the new shader matches CopyKVCacheProgram's existing indirect-dispatch block exactly — same kPopulateIndirectDispatchBufferFn helper, same (seq + tile − 1) / tile formula. No new algorithm is being introduced, just a new call site for the case where CopyKVCache is skipped.
Correct choice of source input: uses total_sequence_length_input[0] (global max, batch-safe) rather than seqlens_k[0] (index 0, unsafe for right-padded batched prefill). Matches what the earlier review round asked for.
Tight gating: PrepareIndirectDispatchProgram only runs when use_indirect_dispatch == true, which itself requires context.IsGraphCaptureEnabled() + seqlen_k != nullptr + total_seqlen != nullptr. Non-graph-capture path is completely unaffected.
The ORT_ENFORCE relaxation in group_query_attention.cc is minimal and correctly conditioned: !GC || kv_empty || past_present_share_buffer_. kv_empty layers don't write to present, so requiring buffer-sharing on them is meaningless.
The test does the right thing. The earlier claude-review round flagged that OpTester can't drive graph capture and so the shader wasn't actually being exercised. The rewritten WebGPU_SharedKV_IndirectDispatchForGraphCapture now uses the real InferenceSession + kEnableGraphCapture=ON + IOBinding path, does a capture run followed by a replay run with different input data written into the same GPU buffers, and verifies the replay output against a CPU reference. That's exactly the coverage refactor safety needs.
CI is green (86 / 86) on the latest commit, including the WebGPU legs.

Minor nits (non-blocking)

Comment wording in group_query_attention.cc: the new comment says "past/present cannot share the same buffer" for kv_empty layers, but in the same branch the code aliases them via present_key = const_cast<Tensor*>(past_key) — they do end up sharing. The truer statement is: "kv_empty layers don't write to present, so past_present_share_buffer_ is not a required precondition." Consider tightening the comment; the code itself is correct.
Test tolerance 0.05f: 5e−2 absolute is fairly generous for a 16-element output. If replay-vs-CPU is expected to be tighter in practice, worth tightening; not blocking.
The 200-line test is worth factoring later. make_gpu_value / update_gpu_value / the manual Graph-API model construction are all things future graph-capture-mode WebGPU tests will want. A small helper in test/util/ would pay for itself the next time someone writes a GC-mode WebGPU test. Not blocking on this PR.
const_cast<Tensor&>(gpu_val.Get<Tensor>()) in update_gpu_value — code smell, but a common one in tests that mutate a preallocated buffer under graph capture (where rebinding is not allowed). Fine as-is.
u32(total_sequence_length_input[0]) in the shader silently wraps if the int32 is negative. In practice total_seqlen is always ≥ 0, so this is safe; a defensive ORT_ENFORCE(parameters.total_sequence_length_ >= 0, ...) on the CPU side would be belt-and-suspenders but isn't required.

…pport Gemma4 (#29236) ## Summary This adds indirect dispatch support for WebGPU flash attention to enable graph capture for Gemma4. When graph capture is on, \`total_seqlen\` is GPU-resident and cannot be read on the CPU, so dispatch group sizes must be computed on GPU. CopyKVCache normally prepares the indirect dispatch buffer as a side effect. For Gemma4 kv_empty layers (shared KV layers that skip CopyKVCache), a dedicated \`PrepareIndirectDispatchProgram\` fills the indirect buffer instead — avoiding \`dispatch(0)\` crashes. - Adds \`PrepareIndirectDispatchProgram\`: a single-thread GPU kernel that reads \`total_sequence_length_input[0]\` and writes 3 x uint32 dispatch dims to \`indirect_buffer\`, matching the logic in \`CopyKVCacheProgram\` - \`use_indirect_dispatch\` is gated on \`context.IsGraphCaptureEnabled()\` — indirect dispatch is only needed when \`total_seqlen\` is GPU-resident; when graph capture is off, CPU-side dispatch sizing works correctly even for kv_empty layers - \`PrepareIndirectDispatchProgram\` takes \`total_sequence_length_input\` (the global max, batch-safe) rather than \`seqlen_k\` (per-batch index 0, unsafe for batch > 1) ## Changes - \`flash_attention.cc\`: - New \`PrepareIndirectDispatchProgram::GenerateShaderCode\`: reads \`total_sequence_length_input[0]\`, computes tile count, calls \`populate_indirect_dispatch_buffer\` — identical logic to \`CopyKVCacheProgram\`'s indirect dispatch block - \`use_indirect_dispatch\` gated on \`seqlen_k != nullptr && total_seqlen != nullptr && context.IsGraphCaptureEnabled()\` - kv_empty path calls \`PrepareIndirectDispatchProgram\` when \`use_indirect_dispatch\` is true (i.e., under graph capture) - \`flash_attention.h\`: Added \`PrepareIndirectDispatchProgram\` class declaration ## Test plan - [x] Verified with Gemma4 INT4 model: ~95-105 tok/s with GC=ON vs ~75 tok/s with GC=OFF - [x] Tested multiple prompts (short/long) with graph capture enabled — no crashes, correct output - [x] Verified warmup (capture run) followed by replay produces consistent results - [x] Unit test: \`WebGPU_SharedKV_IndirectDispatchForGraphCapture\` — exercises the full ORT graph capture simulation path end-to-end: - Model built via the Graph API (opset 17) with all GQA inputs declared as proper graph inputs - All inputs allocated as GPU-resident tensors via \`InferenceSession::GetAllocator\` + \`IOBinding\`; \`total_seqlen\` is a real \`WGPUBuffer\` so \`PrepareIndirectDispatchProgram\` reads it from GPU - WebGPU EP registered with \`kEnableGraphCapture=ON\`; first \`Run\` captures, second \`Run\` replays - Replay output copied GPU→CPU and compared against a CPU reference to verify correctness - Covers the kv_empty branch specifically: \`key\`/\`value\` have \`sequence_length=0\`, triggering the \`PrepareIndirectDispatchProgram\` code path (CopyKVCache is skipped for kv_empty layers) --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

… flag to support Gemma4 (#29392) ## Summary These four ops were forcing CPU fallback when inputs were INT64, preventing WebGPU graph capture. Converted from static macro registration to the factory function pattern (same as Cast/Unsqueeze/Expand/Range) so INT64 type constraints are included when `enable_int64=true`. `enable_int64` is always `true` when `enable_graph_capture=true` — `GetKernelRegistry` calls `RegisterKernels(true, true)` unconditionally in that path — so this fix has no effect on the default (non-graph-capture) execution path. ## Changed ops | Op | Versions | |---|---| | `Equal` | 7–10, 11–12, 13–18, 19+ | | `Sub` | 7–12, 13, 14+ | | `Where` | 9–15, 16+ | | `ReduceSum` | 1–10, 11–12, 13+ | ## Approach Each op follows the established factory function pattern: - `CreateXVersionedKernelInfo<Start, End>(bool enable_int64)` / `CreateXKernelInfo<Since>(bool enable_int64)` - Removed from static `build_kernel_create_info_function_table[]` - Registered in `RegisterKernels()` alongside Cast/Unsqueeze/Expand/Range `ReduceSum` version 13+ retains `InputMemoryType(OrtMemTypeCPUInput, 1)` (axes input must be on CPU) — unchanged from the original registration. `Where` factory functions delegate to `GetOpTypeConstraints(enable_int64, /*enable_bool=*/true)` rather than a local duplicate type list. All three files (`where.cc`, `binary_elementwise_ops.cc`, `reduction_ops.cc`) use stack-allocated `KernelDefBuilder()` consistent with other WebGPU factory functions, and drop the unused `webgpu_execution_provider.h` include. ## Runtime fixes for INT64 inference `BinaryElementwise` (`Sub`, `Equal`) and `Where` required additional shader-level fixes to correctly handle INT64 tensors at inference time: - **Vectorization disabled for INT64**: INT64 is stored as `vec2<u32>` (8 bytes/element) and has no vec4 representation. Vectorization is now disabled when either input is INT64, and `component=1` is used for correct buffer binding. - **Correct dispatch size**: `vec_size` is computed as element count (not rounded-up vec4 count) for INT64 outputs. - **Scalar input path**: The scalar broadcast path no longer applies an extra `.x` dereference on top of `GetByOffset`, which already extracts the `i32` element for INT64. - **Where non-broadcast path**: Non-broadcast INT64 `Where` now uses the element-per-thread shader path instead of the vec4 fast path that would truncate values to i32. - **Cache key**: `is_int64` is included in the `Where` cache hint, preventing float and INT64 shaders from incorrectly sharing a cached entry. - **Shape indices for INT64**: Shape indices are registered for non-broadcast INT64 `Where` so `BroadcastedIndicesToOffset` has the required helpers available. - **Equal INT64 with bool output**: The element-wise path now explicitly reads 4 consecutive INT64 elements per thread and packs them into a `vec4<bool>` before writing to the bool output buffer. Previously, only element 0 was read and broadcast across all 4 positions of the comparison, giving wrong results. ## Testing - All 1174 nodes of a Gemma4 WebGPU decoder model assigned to `WebGpuExecutionProvider` (previously ~20 nodes fell back to CPU due to INT64 Equal/Sub/Where/ReduceSum) - End-to-end graph capture inference: ~87 tok/s on Gemma4-E2B-it INT4 - `USE_WEBGPU`-guarded unit tests added for all four ops with `kEnableInt64=1`: `Sub_int64_webgpu`, `Equal_int64_webgpu`, `Where_int64_webgpu`, `ReduceSum_int64_webgpu` ## Related - Indirect dispatch fix (prerequisite for graph capture): #29236 - onnxruntime/mobius#380 --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

feich-ms requested a review from Copilot June 26, 2026 03:19

Copilot started reviewing on behalf of feich-ms June 26, 2026 03:19 View session

feich-ms added the ep:WebGPU ort-web webgpu provider label Jun 26, 2026

feich-ms marked this pull request as ready for review June 26, 2026 03:23

Copilot AI reviewed Jun 26, 2026

View reviewed changes

feich-ms requested a review from qjia7 June 26, 2026 08:02

feich-ms force-pushed the user/feich/gemma4_webgpu_gc_support branch from 6ae8f1f to 63159eb Compare June 26, 2026 08:31

github-actions Bot reviewed Jun 26, 2026

View reviewed changes

Comment thread onnxruntime/test/contrib_ops/group_query_attention_op_test.cc Outdated

Comment thread onnxruntime/test/contrib_ops/group_query_attention_op_test.cc Outdated

Comment thread onnxruntime/test/contrib_ops/group_query_attention_op_test.cc Outdated

feich-ms marked this pull request as draft June 26, 2026 11:00

feich-ms marked this pull request as ready for review June 29, 2026 06:23

feich-ms requested a review from guschmue June 29, 2026 08:49

feich-ms changed the title ~~WebGPU: Add indirect dispatch for flash attention graph capture~~ WebGPU: Add indirect dispatch for flash attention graph capture to support Gemma4 Jun 29, 2026

feich-ms mentioned this pull request Jun 29, 2026

WebGPU: enable INT64 for Equal/Sub/Where/ReduceSum under enable_int64 flag to support Gemma4 #29392

Merged

feich-ms and others added 8 commits June 30, 2026 10:22

add indirect dispatch support for graph capture

c5f0345

webgpu: clang-format and remove verbose comment in flash attention

e5f62cd

Co-Authored-By: Claude <noreply@anthropic.com>

webgpu: fix RunGQASharedKV call sites for GqaTargetEp API change

4ee58bb

Replace deprecated bool use_cuda/use_webgpu params with GqaTargetEp::kCpu. Co-Authored-By: Claude <noreply@anthropic.com>

feich-ms force-pushed the user/feich/gemma4_webgpu_gc_support branch from 823a239 to 4993603 Compare June 30, 2026 03:49

qjia7 reviewed Jun 30, 2026

View reviewed changes

Comment thread onnxruntime/contrib_ops/webgpu/bert/flash_attention.cc Outdated

feich-ms and others added 2 commits June 30, 2026 15:15

webgpu: inline AppendPopulateIndirectDispatchShader into its only cal…

495aac7

…l site Single-use static helpers add indirection without benefit. Co-Authored-By: Claude <noreply@anthropic.com>

feich-ms requested a review from justinchuby June 30, 2026 08:57

qjia7 requested a review from Copilot June 30, 2026 09:32

Copilot started reviewing on behalf of qjia7 June 30, 2026 09:32 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

qjia7 reviewed Jun 30, 2026

View reviewed changes

Copilot started work on behalf of feich-ms June 30, 2026 13:59 View session

Fix kv_empty WebGPU tests: reuse EP, drop unvalidated present outputs

99c41ee

Copilot finished work on behalf of feich-ms June 30, 2026 14:03

feich-ms requested a review from hariharans29 July 1, 2026 01:16

qjia7 reviewed Jul 1, 2026

View reviewed changes

Comment thread onnxruntime/test/contrib_ops/group_query_attention_op_test.cc Outdated

feich-ms and others added 3 commits July 1, 2026 20:53

webgpu: move #ifdef USE_WEBGPU to wrap the whole GC test, not just body

053c2b9

Co-Authored-By: Claude <noreply@anthropic.com>

qjia7 approved these changes Jul 2, 2026

View reviewed changes

hariharans29 approved these changes Jul 2, 2026

View reviewed changes

justinchuby merged commit d759254 into main Jul 2, 2026
86 checks passed

justinchuby deleted the user/feich/gemma4_webgpu_gc_support branch July 2, 2026 21:53

aprehot mentioned this pull request Jul 7, 2026

[Web] WebGPU EP graph-capture replay misbinds an internal uniform buffer for a static-shape encoder transformer (incorrect outputs) #29594

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

WebGPU: Add indirect dispatch for flash attention graph capture to support Gemma4#29236

WebGPU: Add indirect dispatch for flash attention graph capture to support Gemma4#29236
justinchuby merged 16 commits into
mainfrom
user/feich/gemma4_webgpu_gc_support

feich-ms commented Jun 23, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

qjia7 left a comment

Uh oh!

Uh oh!

hariharans29 commented Jul 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

feich-ms commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

qjia7 left a comment

Choose a reason for hiding this comment

Summary

Findings

Q1 — Test duplication of RunGQASharedKV

Q2 — The new shader is not exercised by either test

Verdict

Uh oh!

Uh oh!

hariharans29 commented Jul 2, 2026

Review: PR #29236 — WebGPU: Add indirect dispatch for flash attention graph capture to support Gemma4

What's good

Minor nits (non-blocking)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

feich-ms commented Jun 23, 2026 •

edited

Loading

Q1 — Test duplication of `RunGQASharedKV`